The subject matter discussed in this section should not be assumed to be prior art merely as a result of its mention in this section. Similarly, a problem mentioned in this section or associated with the subject matter provided as background should not be assumed to have been previously recognized in the prior art. The subject matter in this section merely represents different approaches, which in and of themselves can also correspond to implementations of the claimed technology.
Common methods for running controlled experiments on websites include sophisticated conversion optimization solutions. Conversion optimization includes testing multiple combinations and variations of webpages and webpage elements at the same time. For example, two alternative images, plus two alternative headlines, plus two copy text alternatives, for a total of twenty-seven possible combinations (including the original control versions) may be provided. Thus, conversion optimization introduces a rather complex set of permutations and combinations that need to be analyzed to determine the most effective combination of webpage elements that truly engage the users.
As Big Data plays a more important role in web personalization, the number of data signals, the complexity of rules, and the sheer number of outcomes has increased exponentially. As that happens, human optimization simply cannot be done except perhaps after the fact, where there is little to no opportunity to impact the outcome. Algorithmic optimization is required, but even there, simple linear regression algorithms that can handle linear relationships and correlations may not be able to sufficiently create improved outcomes, given the vast number of data inputs and resulting measurements that have to be processed to predict performance.
In e-commerce, designing user experiences, i.e., webpages and interactions, which convert as many users as possible from casual browsers to paying customers is an important goal. While there are some well-known design principles, including simplicity and consistency, there are also often unexpected interactions between elements of the webpage that determine how well it converts. The same element may work well in one context but not in others. It is often hard to predict the result, and even harder to decide how to improve a given webpage.
An entire industry has emerged to tackle these challenges; it is called conversion rate optimization, or conversion science. The standard method most practitioners use is A/B testing, i.e., designing two different versions of the same webpage, showing them to different users, and collecting statistics on how well they each convert. This process allows incorporating human knowledge about the domain and conversion optimization into the design, and then testing their effect. After observing the results, new designs can be compared and gradually improved. The A/B testing process is difficult and time-consuming: only a very small fraction of webpage designs can be tested in this way and subtle interactions in the design may simply be missed completely.
Machine learning systems are utilized to run tests where many variables with very complex relationships between them are involved in determining outcomes. Machine learning systems typically attempt to learn from the data to figure out the formula, rather than to try to figure out a formula to begin with, given that the relationships between the variables may be too complex to determine the algorithm in advance. Therefore, with so many variables at play in conversion optimization, very sophisticated algorithms are desirable that utilize machine learning, artificial intelligence, and other non-linear algorithms to make predictions about outcomes based on learning from large data sets of inputs.